Skip to content

MON-4136: set up the PrometheusPossibleNarrowSelectors alert#2554

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
machine424:smooth-prom3
Mar 11, 2025
Merged

MON-4136: set up the PrometheusPossibleNarrowSelectors alert#2554
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
machine424:smooth-prom3

Conversation

@machine424

Copy link
Copy Markdown
Contributor
  • I added CHANGELOG entry for this change.
  • No user facing changes, so no entry in CHANGELOG was needed.

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 8, 2025
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 8, 2025
Comment thread jsonnet/rules.libsonnet Outdated
Comment thread jsonnet/rules.libsonnet
Comment thread assets/cluster-monitoring-operator/prometheus-rule.yaml Outdated
Comment thread pkg/manifests/manifests.go Outdated
@machine424 machine424 force-pushed the smooth-prom3 branch 4 times, most recently from d1d66f8 to 95f4a41 Compare January 15, 2025 17:58
@machine424 machine424 changed the title WIP: feat: set up alerts for a smooth Prom3 upgrade WIP: set up the PrometheusPossibleNarrowSelectors alert Jan 15, 2025
@machine424

machine424 commented Jan 15, 2025

Copy link
Copy Markdown
Contributor Author

/testwith openshift/cluster-monitoring-operator/master/e2e-agnostic-operator openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-upgrade openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-upgrade openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-single-node openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-techpreview openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-hypershift-conformance openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611

@openshift-ci

openshift-ci Bot commented Jan 15, 2025

Copy link
Copy Markdown
Contributor

@machine424, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-agnostic-operator' not found in injected config

@openshift-ci

openshift-ci Bot commented Jan 15, 2025

Copy link
Copy Markdown
Contributor

@machine424, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-aws-ovn-single-node' not found in injected config

@openshift-ci

openshift-ci Bot commented Jan 15, 2025

Copy link
Copy Markdown
Contributor

@machine424, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-aws-ovn-techpreview' not found in injected config

@openshift-ci

openshift-ci Bot commented Jan 15, 2025

Copy link
Copy Markdown
Contributor

@machine424, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-hypershift-conformance' not found in injected config

@machine424

Copy link
Copy Markdown
Contributor Author

/testwith openshift/cluster-monitoring-operator/master/e2e-agnostic-operator openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-upgrade openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-single-node openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-techpreview openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-hypershift-conformance openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611

1 similar comment
@machine424

Copy link
Copy Markdown
Contributor Author

/testwith openshift/cluster-monitoring-operator/master/e2e-agnostic-operator openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-upgrade openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-single-node openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-aws-ovn-techpreview openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611
/testwith openshift/cluster-monitoring-operator/master/e2e-hypershift-conformance openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611

@machine424

machine424 commented Jan 16, 2025

Copy link
Copy Markdown
Contributor Author

@openshift-ci

openshift-ci Bot commented Jan 16, 2025

Copy link
Copy Markdown
Contributor

@machine424, testwith: Error processing request. ERROR:

could not determine job runs: invalid format for additional PR: openshift/openshift/cluster-authentication-operator#752

@machine424

Copy link
Copy Markdown
Contributor Author

/payload-job-with-prs periodic-ci-openshift-release-master-nightly-4.19-e2e-rosa-sts-hypershift-ovn openshift/prometheus#227 openshift/cluster-kube-apiserver-operator#1784 openshift/cluster-openshift-apiserver-operator#611 openshift/cluster-authentication-operator#752 #2554

@openshift-ci

openshift-ci Bot commented Jan 16, 2025

Copy link
Copy Markdown
Contributor

@machine424: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.19-e2e-rosa-sts-hypershift-ovn

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/50647e70-d3fb-11ef-84ae-020c1e634813-0

@openshift-ci

openshift-ci Bot commented Jan 16, 2025

Copy link
Copy Markdown
Contributor

@machine424: This PR was included in a payload test run from #2554
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.19-e2e-rosa-sts-hypershift-ovn

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/50647e70-d3fb-11ef-84ae-020c1e634813-0

@openshift-ci-robot

openshift-ci-robot commented Feb 10, 2025

Copy link
Copy Markdown
Contributor

@machine424: This pull request references MON-4136 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

Details

In response to this:

  • I added CHANGELOG entry for this change.
  • No user facing changes, so no entry in CHANGELOG was needed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@machine424

Copy link
Copy Markdown
Contributor Author

/test ?

@openshift-ci

openshift-ci Bot commented Feb 12, 2025

Copy link
Copy Markdown
Contributor

@machine424: The following commands are available to trigger required jobs:

/test e2e-agnostic-operator
/test e2e-aws-ovn
/test e2e-aws-ovn-techpreview
/test e2e-aws-ovn-upgrade
/test e2e-hypershift-conformance
/test generate
/test go-fmt
/test golangci-lint
/test images
/test jsonnet-fmt
/test okd-scos-images
/test rules
/test shellcheck
/test unit
/test vendor
/test verify

The following commands are available to trigger optional jobs:

/test e2e-aws-ovn-single-node
/test okd-scos-e2e-aws-ovn
/test versions

Use /test all to run all jobs.

Details

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@machine424

Copy link
Copy Markdown
Contributor Author

/retest

@machine424

Copy link
Copy Markdown
Contributor Author

/payload-with-prs 4.19 nightly blocking openshift/thanos#158

@openshift-ci

openshift-ci Bot commented Mar 4, 2025

Copy link
Copy Markdown
Contributor

@machine424: trigger 12 job(s) of type blocking for the nightly release of OCP 4.19

  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-upgrade-ovn-single-node
  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.19-e2e-azure-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.19-upgrade-from-stable-4.18-e2e-gcp-ovn-rt-upgrade
  • periodic-ci-openshift-hypershift-release-4.19-periodics-e2e-aws-ovn-conformance
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-aws-ovn-serial
  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview
  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview-serial
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-aws-driver-toolkit
  • periodic-ci-openshift-release-master-nightly-4.19-fips-payload-scan
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-bm
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-ipv6

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/64699ce0-f8ca-11ef-8da6-947ded54a337-0

@machine424

Copy link
Copy Markdown
Contributor Author

/payload-with-prs 4.19 ci blocking openshift/thanos#158

@openshift-ci

openshift-ci Bot commented Mar 4, 2025

Copy link
Copy Markdown
Contributor

@machine424: trigger 4 job(s) of type blocking for the ci release of OCP 4.19

  • periodic-ci-openshift-release-master-ci-4.19-upgrade-from-stable-4.18-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.19-upgrade-from-stable-4.18-e2e-azure-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.19-e2e-gcp-ovn-upgrade
  • periodic-ci-openshift-hypershift-release-4.19-periodics-e2e-aws-ovn

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/6cd07de0-f8ca-11ef-87ff-500987415ff7-0

@machine424

Copy link
Copy Markdown
Contributor Author

I should have waited a little longer

�[36mINFO�[0m[2025-03-04T07:31:24Z] Requesting a release from https://amd64.ocp.releases.ci.openshift.org/api/v1/releasestream/4.19.0-0.nightly/latest?rel=1 
�[36mINFO�[0m[2025-03-04T07:31:24Z] Resolved release initial to registry.ci.openshift.org/ocp/release:4.19.0-0.nightly-2025-02-28-132549 

Which doesn't ship v3

@machine424

Copy link
Copy Markdown
Contributor Author

/payload-with-prs 4.19 ci blocking openshift/thanos#158

@machine424

Copy link
Copy Markdown
Contributor Author

/payload-with-prs 4.19 nightly blocking openshift/thanos#158

@openshift-ci

openshift-ci Bot commented Mar 4, 2025

Copy link
Copy Markdown
Contributor

@machine424: trigger 12 job(s) of type blocking for the nightly release of OCP 4.19

  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-upgrade-ovn-single-node
  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.19-e2e-azure-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.19-upgrade-from-stable-4.18-e2e-gcp-ovn-rt-upgrade
  • periodic-ci-openshift-hypershift-release-4.19-periodics-e2e-aws-ovn-conformance
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-aws-ovn-serial
  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview
  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn-techpreview-serial
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-aws-driver-toolkit
  • periodic-ci-openshift-release-master-nightly-4.19-fips-payload-scan
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-bm
  • periodic-ci-openshift-release-master-nightly-4.19-e2e-metal-ipi-ovn-ipv6

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/2d6e8b30-f8f5-11ef-86d5-fe8009f34d88-0

@openshift-ci

openshift-ci Bot commented Mar 4, 2025

Copy link
Copy Markdown
Contributor

@machine424: trigger 4 job(s) of type blocking for the ci release of OCP 4.19

  • periodic-ci-openshift-release-master-ci-4.19-upgrade-from-stable-4.18-e2e-aws-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.19-upgrade-from-stable-4.18-e2e-azure-ovn-upgrade
  • periodic-ci-openshift-release-master-ci-4.19-e2e-gcp-ovn-upgrade
  • periodic-ci-openshift-hypershift-release-4.19-periodics-e2e-aws-ovn

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/2940e710-f8f5-11ef-99e9-6a09560173d5-0

@jan--f

jan--f commented Mar 7, 2025

Copy link
Copy Markdown
Contributor

/lgtm
I'm not entirely sure we should put links to Jira in an alert, but I have no better suggestion.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Mar 7, 2025
@openshift-ci

openshift-ci Bot commented Mar 7, 2025

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jan--f, machine424

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@machine424

Copy link
Copy Markdown
Contributor Author

I'm not entirely sure we should put links to Jira in an alert, but I have no better suggestion.

Yes, it's temp, once/If we decide to ship it, we'll take the time to add a runbook, let's see how it performs.

@machine424

Copy link
Copy Markdown
Contributor Author

/hold cancel

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 11, 2025
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 884f872 and 2 for PR HEAD 187d9d7 in total

@openshift-ci

openshift-ci Bot commented Mar 11, 2025

Copy link
Copy Markdown
Contributor

@machine424: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-single-node 187d9d7 link false /test e2e-aws-ovn-single-node
ci/prow/versions 187d9d7 link false /test versions

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 884f872 and 2 for PR HEAD 187d9d7 in total

@openshift-merge-bot openshift-merge-bot Bot merged commit 5df10b7 into openshift:main Mar 11, 2025
@openshift-bot

Copy link
Copy Markdown
Contributor

[ART PR BUILD NOTIFIER]

Distgit: cluster-monitoring-operator
This PR has been included in build cluster-monitoring-operator-container-v4.19.0-202503112209.p0.g5df10b7.assembly.stream.el9.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants